Network-assisted scanning of a surrounding environment

ABSTRACT

A system, method and architecture for network-assisted scanning of a surrounding environment. In one example arrangement, the system is operative for receiving real world object identification and spatial mapping data relative to a plurality of real world scenarios sensed by sensing points, each operating as a collection agent configured to collect environmental data in association with a plurality of sensors. The data may be prioritized responsive to at least one of a determination of RAN resource capacity available to the collection agent, e.g., compute resources, storage resources, radio transmission resources, cost of the transmission capacity available to the collection agent, etc. in view of urgency/relevance of the data. Responsive to prioritizing the environmental data, one or more pieces of the environmental data may be transmitted to a nearest edge compute location and/or a cloud-based datacenter network node.

PRIORITY AND CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims priority based upon the following priorUnited States provisional patent application(s): (i) “NETWORK-ASSISTEDSCANNING OF A SURROUNDING ENVIRONMENT,” Application No. 62/559,749,filed Sep. 18, 2017, in the name(s) of Vladimir Katardjiev, JulienForgeat, Meral Shirazipour and Dmitri Krylov; each of which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure generally relates to communication networks. Moreparticularly, and not by way of any limitation, the present disclosureis directed to a system, method and architecture for facilitatingnetwork-assisted scanning of surrounding environments to obtain highfidelity geospatial mapping information and using the same in anaugmented/mixed reality (AR/MR) environment.

BACKGROUND

Increasingly, augmented and virtual reality (AR/VR) are becoming morethan gaming environments, with companies finding enterprise potential inthe technology in a host of applications. One of the goals of theindustry is to replace conventional user interfaces such as keyboards,displays, etc. with new paradigms for human-machine communication andcollaboration, thereby facilitate a major shift in user engagement inAR/VR environments. Accordingly, the enterprise potential of AR/VRtechnology continues to grow as companies are constantly exploring newuse cases beyond pilot or “one-off” applications.

Mixed reality (MR) represents a further advance where both AR and realworld environments may be merged in additional enhancements to providericher user experiences. As the trends in AR/VR/MR deployment continueto grow apace, interest in providing enhanced presentation of varioustypes of data, e.g., vehicular navigation data, etc., by way of digital“real estate” in AR/VR/MR environments has also grown concomitantly,albeit potentially within the constraints of efficient bandwidthutilization and optimization in an AR-supported network. Relatedly,whereas there have been several advances in the field of autonomousvehicles and navigation systems, significant lacunae continue to exist,as will be set forth hereinbelow.

SUMMARY

The present patent disclosure is broadly directed to systems, methods,apparatuses, devices, and associated non-transitory computer-readablemedia and network architecture for facilitating network-assistedscanning of surrounding environments by efficient use of radio networkcapacity. In one aspect, an example method includes, inter alia,collecting environmental data by a collection agent operating with aplurality of sensors configured to sense data relating to a surroundingenvironment and prioritizing the environmental data responsive to atleast one of a determination of cellular radio access network (RAN)resource capacity available to the collection agent, including but notlimited to: compute resources, storage resources, radio transmissionresources, cost of the cellular RAN resource capacity available to thecollection agent, and/or relative urgency/relevance of an event withinthe environment to which at least a portion of the environmental datapertains, among others. Responsive to prioritizing the environmentaldata, one or more pieces of the environmental data may be transmitted toa nearest edge compute location (e.g., at an edge network nodeassociated with a cellular network) and/or a cloud-based datacenternetwork node. Where cellular connectivity is insufficient, other modesof connectivity (e.g., WiFi) may be applied.

In another aspect, the present patent application is also directed tosystems, methods, apparatuses, devices, and associated non-transitorycomputer-readable media and network architecture for facilitating usageof detailed road data with respect to providing precise drivingrecommendations in a vehicular use case scenario involving manual and/orautonomous control. An example method for facilitating highly assisteddriving (HAD) in a vehicle based on geospatial sensing of an environmentin which the vehicle is disposed comprises, inter alia, receivingenvironmental data from one or more collection agents, each operatingwith a plurality of sensors configured to sense data relating to theenvironment, wherein at least one collection agent is operative inassociation with the vehicle for providing vehicular condition data andvehicular navigation data. Responsive to the vehicularcondition/navigation data, the vehicle positioning and vehicle type maybe calibrated. Road condition information and/or obstacle conditioninformation relating to a road segment the vehicle is traversing mayalso be obtained. A suitable racing line path over the road segment isdetermined, e.g., responsive to optimizing comfort and safety in view ofroad conditions and given vehicle type, direction, speed, etc. In onevariation, the optimal racing line path may be presented via a display,e.g., an AR/MR display, associated with the vehicle.

In a further aspect, an embodiment of a system, apparatus, or networkplatform is disclosed which comprises, inter alia, suitable hardwaresuch as processors and persistent memory having program instructions forexecuting an embodiment of the methods set forth herein. In anotheraspect, an embodiment of a user equipment (UE) device having acollection agent configured to sense and report environmental sensorydata with respect to an ambient environment is disclosed. In stillfurther aspects, one or more embodiments of a non-transitorycomputer-readable medium or distributed media containingcomputer-executable program instructions or code portions stored thereonare disclosed for performing one or more embodiments of the methods ofthe present invention when executed by a processor entity of a networknode, apparatus, system, network element, subscriber device, and thelike, mutatis mutandis. Further features of the various embodiments areas claimed in the dependent claims.

Benefits flowing from an embodiment of the present invention may includebut not limited to one or more of the following: localizedprocessing/encoding to ease the burden on network bandwidth; dataretrieval scheduling with suitable prioritization, e.g., policy-based,operator-configured, etc., for throttling bandwidth usage; lower costupdates to high fidelity mapping databases, resulting in increasedspatial resolution in an economical way; and providing a scalablecrowd-sourcing architecture for efficient sensory data collection as thenumber of consumers, sensing points, vehicles, connectivity options, andthe like continue to grow. Additional benefits and advantages of theembodiments will be apparent in view of the following description andaccompanying Figures.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present disclosure are illustrated by way of example,and not by way of limitation, in the Figures of the accompanyingdrawings in which like references indicate similar elements. It shouldbe noted that different references to “an” or “one” embodiment in thisdisclosure are not necessarily to the same embodiment, and suchreferences may mean at least one. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to effect such feature, structure, or characteristicin connection with other embodiments whether or not explicitlydescribed.

The accompanying drawings are incorporated into and form a part of thespecification to illustrate one or more exemplary embodiments of thepresent disclosure. Various advantages and features of the disclosurewill be understood from the following Detailed Description taken inconnection with the appended claims and with reference to the attacheddrawing Figures in which:

FIG. 1 depicts an example architecture for facilitating network-assistedenvironmental scanning for purposes of one or more embodiments of thepresent invention;

FIGS. 2A and 2B are flowcharts of various blocks, steps and/or acts thatmay be (re)combined in one or more arrangements, with or withoutadditional flowcharts of the present disclosure, for facilitatingenvironmental and ambient sensory recognition according to one or moreembodiments of the present patent application;

FIGS. 3A-3C depict a functional block diagram illustrative of variouscomponents, blocks, modules network elements and/or apparatuses that maybe combined into an example implementation of the network architectureof FIG. 1 according to one embodiment;

FIGS. 4A-4B are a flowchart of various blocks, steps and/or acts thatmay be (re)combined in one or more arrangements, with or withoutadditional flowcharts of the present disclosure, for facilitatingoptimized use of network capacity for purposes of an embodiment of thepresent patent application;

FIG. 5 depicts an example implementation of the network architecture ofFIG. 1 that illustrates one or more sensing points and one or moreconsumption points according to an embodiment of the present patentapplication;

FIG. 6 is an example wireless network environment for facilitatingresource-based environmental data collection and processing for purposesof one or more embodiments of the present invention;

FIG. 7 depicts a block diagram of a computer-implemented apparatus thatmay be (re)configured and/or (re)arranged as a platform, node or elementat an edge network location, a core network location, and/or acloud-based datacenter location according to an embodiment of thepresent patent disclosure;

FIG. 8 depicts a block diagram of a user equipment (UE) node having acollection agent according to an embodiment of the present patentdisclosure;

FIG. 9 depicts a diagram showing a manual/autonomous vehicle havingvarious sensors and on-board monitoring devices operative with acollection agent integrated within the vehicle or in conjunction with aUE device placed within the vehicle for purposes of an embodiment of thepresent patent disclosure;

FIGS. 10 and 11 depict block diagrams of components of a vehicle havinga collection agent for facilitating highly assisted driving (HAD)according an embodiment of the present patent disclosure;

FIG. 12 depicts an example geographical region/area having a roadnetwork traversed by a plurality of manual/autonomous vehicles operativeto collect and process environmental sensory data in accordance with theteachings of the present patent disclosure;

FIGS. 13A and 13B depict a functional block diagram illustrative ofvarious components, blocks, modules, network elements and/or apparatusesthat may be configured as a system operative in association with thenetwork architecture of FIGS. 3A-3C for facilitating highly assisteddriving based on up-to-date environmental data according to oneembodiment;

FIGS. 14A-1 to 14A-3, 14B and 14C are flowcharts of various blocks,steps and/or acts that may be (re)combined in one or more arrangements,with or without additional flowcharts of the present disclosure, forfacilitating highly assisted driving using the system of FIGS. 13A-13B;

FIG. 15 is illustrative of a vehicle with AR display for showing arouting path or racing line presented based on up-to-date environmentaldata according to an embodiment of the present invention; and

FIG. 16 is a flowchart of various blocks, steps and/or acts that may be(re)combined in one or more arrangements, with or without additionalflowcharts of the present disclosure, for facilitating a process inassociation with the system of FIGS. 13A-13B according to an embodimentof the present invention.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forthwith respect to one or more embodiments of the present patentdisclosure. However, it should be understood that one or moreembodiments may be practiced without such specific details. In otherinstances, well-known circuits, subsystems, components, structures andtechniques have not been shown in detail in order not to obscure theunderstanding of the example embodiments. Accordingly, it will beappreciated by one skilled in the art that the embodiments of thepresent disclosure may be practiced without such specific components. Itshould be further recognized that those of ordinary skill in the art,with the aid of the Detailed Description set forth herein and takingreference to the accompanying drawings, will be able to make and use oneor more embodiments without undue experimentation.

Additionally, terms such as “coupled” and “connected,” along with theirderivatives, may be used in the following description, claims, or both.It should be understood that these terms are not necessarily intended assynonyms for each other. “Coupled” may be used to indicate that two ormore elements, which may or may not be in direct physical or electricalcontact with each other, co-operate or interact with each other.“Connected” may be used to indicate the establishment of communication,i.e., a communicative relationship, between two or more elements thatare coupled with each other. Further, in one or more example embodimentsset forth herein, generally speaking, an element, component or modulemay be configured to perform a function if the element is capable ofperforming or otherwise structurally arranged or programmed undersuitable executable code to perform that function.

As used herein, a network element, platform or node may be comprised ofone or more pieces of service network equipment, including hardware andsoftware that communicatively interconnects other equipment on a network(e.g., other network elements, end stations, etc.), and is adapted tohost one or more applications or services with respect to a plurality ofsubscribers and associated client devices as well as other endpoints andInternet-of-Things (IoT)-based entities, each executing suitable clientapplications configured to sense/collect various types of data,information, measurements, etc. for facilitating real-time or nearreal-time geospatial mapping or imaging of physical environments. Assuch, some network elements may be disposed in a cellular wireless orsatellite telecommunications network, or a broadband wireline network,whereas other network elements may be disposed in a publicpacket-switched network infrastructure (e.g., the Internet or worldwideweb, also sometimes referred to as the “cloud”), private packet-switchednetwork infrastructures such as Intranets and enterprise networks, aswell as service provider network infrastructures, any of which may spanor involve a variety of access networks and core networks in ahierarchical arrangement. In still further arrangements, one or morenetwork elements may be disposed in cloud-based platforms or datacentershaving suitable equipment running virtualized functions or applicationsrelative to various types of environmental data processing, geospatialimaging/mapping and rendering, which may include generation of AR/MRcontent, audio/video/graphics content, computer-generated imaging (CGI)content or holographic content, etc., suitable for presentation/displayusing a variety of technologies.

Example end stations, client devices or on-board monitoring units invehicles (e.g., manual and/or autonomous vehicles) or any IoT-entityadapted to facilitate sensory data collection may comprise any deviceconfigured to execute, inter alia, real-time data collection inassociation with a variety of sensors and upload the data to a nearestedge node location e.g., via a suitable access network or edge networkarrangement based on a variety of access technologies, standards andprotocols, as will be described in detail below. Furthermore, an exampleclient device may also cooperate with AR/MR devices including such as,e.g., a Google Glass device, Microsoft HoloLens device, etc., as well asholographic computing devices, which may or may not be deployed inassociation with additional local hardware such as networked or localgaming engines/consoles (such as Wii®, Play Station 3®, etc.), portablelaptops, netbooks, palm tops, tablets, phablets, mobile phones,smartphones, multimedia/video phones, mobile/wireless user equipment,portable media players, smart wearables such as smart watches, goggles,digital gloves, and the like. Further, some client devices may alsoaccess or consume other content/services (e.g., non-AR/MR) provided overbroadcast networks (e.g., cable and satellite networks) as well as apacket-switched wide area public network such as the Internet viasuitable service provider access networks. In a still further variation,some client devices or subscriber end stations may also access orconsume content/services provided on virtual private networks (VPNs)overlaid on (e.g., tunneled through) the Internet.

One or more embodiments of the present patent disclosure may beimplemented using different combinations of software, firmware, and/orhardware in one or more modules suitably programmed and/or configured.Thus, one or more of the techniques shown in the Figures (e.g.,flowcharts) may be implemented using code and data stored and executedon one or more electronic devices or nodes (e.g., a subscriber clientdevice or end station, a network element, etc.). Such electronic devicesmay store and communicate (internally and/or with other electronicdevices over a network) code and data using computer-readable media,such as non-transitory computer-readable storage media (e.g., magneticdisks, optical disks, random access memory, read-only memory, flashmemory devices, phase-change memory, etc.), transitory computer-readabletransmission media (e.g., electrical, optical, acoustical or other formof propagated signals—such as carrier waves, infrared signals, digitalsignals), etc. In addition, such network elements may typically includea set of one or more processors coupled to one or more other components,such as one or more storage devices (e.g., non-transitorymachine-readable storage media) as well as storage database(s), userinput/output devices (e.g., a keyboard, a touch screen, a pointingdevice, and/or a display), and network connections for effectuatingsignaling and/or bearer media transmission. The coupling of the set ofprocessors and other components may be typically through one or morebuses and bridges (also termed as bus controllers), arranged in anyknown (e.g., symmetric/shared multiprocessing) or heretofore unknownarchitectures. Thus, the storage device or component of a givenelectronic device or network element may be configured to store codeand/or data for execution on one or more processors of that element,node or electronic device for purposes of implementing one or moretechniques of the present disclosure.

Referring now to the drawings and more particularly to FIG. 1, depictedtherein is an example network environment 100 for facilitatingnetwork-assisted environmental scanning for purposes of one or moreembodiments of the present invention. A plurality of collection agents(CAs) 102-1 to 102-N may be executed on or operative in association withvarious end stations, mobile communications devices, subscriber/userequipment (S/UE) including smart handheld devices such as tablets,phablets, gaming devices, location/navigation devices, smart wearabledevices, manual/autonomous vehicles, as well as variousInternet-of-Things (IoT)-enabled devices, appliances and entities,wherein suitable service logic of a collection agent may be configuredto interoperate with one or more heterogeneous sensors for gatheringenvironmental data with respect to respective physical surroundings,adjacent spaces and structures, and/or nearby vicinities using any knownor heretofore unknown sensing technologies. By way of example, smartwearable devices 106 are representative of smart electronic devices thatcan be worn by humans as implants, fashion accessories, health/activitymonitors, etc. Mobile communications devices 108 are exemplary of cellphones and/or smartphones that can interface with suitable RANinfrastructures using a variety of radio communications technologies.Manual/autonomous vehicles 110 may comprise any type of terrestrialvehicles (e.g., automobiles, vans, trucks, etc.) that may be driven inmanual mode by humans or automatically by way of autonomous control(i.e., self-driving or driverless), or a combination thereof, which mayinclude a vehicle control unit having an integrated collection agentfunctionality, or an on-board interface configured to communicate withan associated UE device having the collection agent functionality.Further, autonomous vehicles 110 may also include unmanned aerialvehicles (UAVs or drones), unmanned watercraft such as boats,submersibles, amphibious vehicles, etc. with appropriately configuredcollection agents executing thereon in an example embodiment of thepresent invention. IoT-enabled entities 112 may comprise smartappliances, smart buildings, and smart infrastructure components suchas, e.g., smart traffic/road sensors and congestion monitors, trafficlights, lamp posts, street signposts, telephone posts, electric utilitypoles, and the like, each having a collection agent module operativewith suitable sensors for facilitating high-fidelity scanning ofadjacent spaces, surroundings, etc., including real-time traffic/roadconditions, weather conditions, and the like, at a high granularitylevel.

Skilled artisans will recognize that regardless of where a collectionagent is disposed or whether it is integrated within a vehicle oroperative in association with smart infrastructural elements of theenvironment, the collection agent and/or associated hardware/firmwarecomponents, (e.g., including one or more sensors), may be considered asa “sensing point” for purposes of an embodiment of the presentinvention. Furthermore, the environmental data gathered, sensed,monitored, measured, detected, and/or determined by the associatedsensors may be broadly grouped into internal sensory data and externalsensory data, wherein internal sensory data may relate to the internalenvironment of a device, apparatus or vehicle, etc., with which thecollection agent and sensors are associated. On the other hand, externalsensory data may relate to the external environment, e.g., outside thevehicle in which the collection agent is disposed. By way ofillustration, a sensor monitoring the braking system of a vehicle maygenerate internal sensory data (e.g., suspension, traction control,etc.) while an imaging sensor (e.g., a camera) and/or a proximity sensorof the vehicle may generate external sensory data relating topedestrians, other vehicles, obstacles including animate and inanimateobjects or fixed/mobile objects, etc. In a further classificationscheme, sensors can be transport related sensors such as e.g.,accelerometer, position data sensors (e.g., GPS), sound data sensors,(e.g., microphone or other audio sensor or sonar (sound navigation andranging)) and visual data sensors (e.g., cameras, optical sensors suchas Lidar or Light Detection and Ranging), or general environment sensorsconfigured as mobile device sensors, wearable sensors, infrastructuresensors, and the like. In general, therefore, collection agents 102-1 to102-N and associated sensors may be deployed in a variety of sensingpoint arrangements, collectively shown as sensing point assemblage 104,operative with respect to a geographical area, location, region, etc.

Sensing points 104 may be configured with suitable network interfaces(not specifically shown in FIG. 1) for communicating with one or morewireless communications networks, public/private data communicationsnetworks, etc., collectively shown as networks 116, via appropriateaccess infrastructures 114 using associated technologies, standards andprotocols. By way of example, sensing points 104 may communicate withnetworks 116 using one or more wireless technologies involving IEEE802.11b, IEEE 802.11a, IEEE 802.11g, IEEE 802.11p, HiperLan and HiperLanII standards, Wi-Max standard, OpenAir standard, Bluetooth standard, anEnhanced Data Rates for Global System for Mobile Communications (GSM)Evolution (EDGE) network technology, a 3^(rd)/4^(th)/5^(th) Generationnetwork technology, Long Term Evolution (LTE) technology, High-SpeedUplink Packet Access (HSUPA) technology, Evolved High-Speed PacketAccess (HSPA) technology, an Integrated Digital Enhanced Network (IDEN)technology, a Code Division Multiple Access (CDMA) network technology, aUniversal Mobile Telecommunications System (UMTS) network technology, aUniversal Terrestrial Radio Access Network (UTRAN) technology, an All-IPNext Generation Network (NGN) technology, an IP Multimedia Subsystem(IMS) technology, and a satellite telephony network technology, etc.

To facilitate high fidelity, high definition geospatial mapping/modelingof the sensed environment, sensing points 104 are preferably configuredto operate with appropriate service logic (network-centric,device-centric, or a combination thereof) to prioritize the sensedenvironmental data and transmit the data to an environmental dataprocessing (EDP) platform 120 coupled to the network 116. In oneembodiment, EDP platform 120 may be disposed at an edge network locationof the cellular radio networks serving the respective sensing points. Inan additional or alternative arrangement, EDP platform 120 may bedisposed at a cloud-based datacenter. As will be set forth in additionaldetail below, the environmental data may be prioritized responsive to,for example, at least one of a determination of cellular RAN resourcecapacities available to the respective sensing points or collectionagents, cost of the cellular RAN resource capacity available to thecollection agents, and/or relative urgency/relevancy of the sensed datawith respect to the events observed within the environment.

Irrespective of where an embodiment of EDP platform 120 is deployed, thereceived environmental data may be collated, processed and analyzed forgenerating suitable output data that may be consumed by one or more dataconsumption points 118-1 to 118-K, which may be individuals, entities,manual/autonomous vehicles, robotic equipment, etc. within theenvironment and/or other entities or third-parties, includinggovernmental agencies or public safety answering point (PSAP) networks,and the like. Further, output data may be provided to the dataconsumption points 118-1 to 118-K in a variety of modalities, e.g.,push, pull, push/pull hybrid, on-demand, etc., as a subscription-basedservice or as part of a zero-rated service. In one embodiment, thereceived environmental data may be processed using various techniquessuch as expert systems, machine learning, artificial intelligence,adaptive neural networks, pattern recognition, fuzzy logic, heuristicanalysis, statistical analysis, Big Data analytics, etc. In a furtherarrangement, the received environmental data may be processed inconjunction with data from other sources, e.g., third-party dataaggregators, network operators, public/governmental agencies, and thelike.

FIGS. 2A and 2B are flowcharts of various blocks, steps and/or acts thatmay be (re)combined in one or more arrangements, with or withoutadditional flowcharts of the present disclosure, for facilitatingenvironmental and ambient sensory recognition according to one or moreembodiments of the present patent application. In particular, process200A is exemplary of a network-assisted scanning of a surroundingenvironment operative in an embodiment of the network architecture 100set forth above. At block 202, collection/sensing of internal andexternal environmental data, e.g., sensory data, may performed bycollection agents in association with one or more sensors, e.g., IoTentities, mobile phones, smart wearables, vehicular/on-board telematics,etc. At block 204, a process of evaluation and prioritization of datamay be executed in view of available network resources, e.g., includingbut not limited to: compute resources, storage resources, radiotransmission resources, cost constraints, criticality/significance ofobservations and events, network/device related timing constraints, etc.(e.g., responsive to network-based service logic in association with thecollection agents). Responsive to the prioritization/evaluation, the rawsensory data may be uploaded to one or more edge network/compute nodes,cloud-based datacenter nodes, and/or data consumption agents (e.g.,Vehicle-to-Vehicle (V2V) and/or Vehicle-to-Infrastructure (V2I, orsomewhat synonymously, V2X) communications in one embodiment), as setforth at block 206. In a further arrangement, the raw sensory data mayalso be locally processed and/or cached (i.e., at the sensing point) forfacilitating optimal utilization of the available cellular RAN resources(e.g., depending on signal strength, data compression, deltacompression, etc.). In another arrangement, the sensory data may betransmitted by leveraging available lower cost servicing provided bymobile network operators (MNOs), mobile virtual network operators(MVNOs) and/or Internet service providers (ISPs). Examples of such lowercost serving may include zero-rating, toll-free data, and/or sponsoreddata, wherein the operator does not charge end users for uploadingsensory data pursuant to a specific differentiated service application(e.g., as a bundled client data service) in limited or metered dataplans of the end users. Skilled artisans will recognize that from theend user's and/or vehicle manufacturer's perspective, contributing thesensory data may be seen as “optional,” and therefore may be amenable tosharing/uploading the sensory data. Otherwise, if the parties arerequired to pay for the bandwidth, it may deter data contributions.Accordingly, uploading the sensory data pursuant to a zero-rating schemeis particularly advantageous in an embodiment of the present invention.

Process 200B shown in FIG. 2B is broadly representative of a processthat may be executed in association with an EDP platform according to anembodiment of the present invention. At block 222, various pieces ofenvironmental data may be received from a plurality of sensing points,e.g., individuals, intelligent/connected robotic equipment, smartinfrastructural components, manual/autonomous vehicles, IoT-basedentities, etc. As noted above, the environmental data may betransmitted/received upon prioritization for optimizing available radioresources. Also, in one example implementation, the environmental datamay be transmitted/received pursuant to a crowd-sourcing arrangement orsourcing model where the information may be obtained by enlisting alarge number of individuals, vehicle operators such as taxicab drivers,or corporate entities operating a fleet of autonomous vehicles, etc.that agree to use a collection agent for a fee, benefit, value and/orother monetary/nonmonetary consideration. An analytics engine executingat the EDP platform is operative to apply techniques such as, e.g.pattern recognition, artificial intelligence (AI), fuzzy logic, neuralnetworks, machine learning, big data analytics, etc., as noted above,for generating 2D/3D high definition (HD) maps, providing real-time mapupdates (delta updating), integration with historical databases, vehiclecalibration/placement, route planning, generation of AR/MR-friendly dataoutput formats, and the like (block 224). Suitably rendered output maybe provided to one or more consumption points (block 226), e.g.,including manual/autonomous vehicles employing technologies such ashighly assisted driving (HAD), advanced driver assistance system (ADAS)control, connected driving, etc. using various user/machine interfacetechnologies including, e.g., AR/MR, Heads-up Display (HUD), On-boardDisplay (OBD) output, etc. As will be further exemplified below,additional use cases for the output data may involve, withoutlimitation, the following under the broad category of real-timeambient-assisted awareness: real-time AR/MR rendering for assistingcyclists or pedestrians in some fashion, e.g., highlight various roaddangers/obstacles, including animate and inanimate objects; real-timeAR/MR rendering for helping tourists discover local surroundings' statuswith images and sounds (e.g., local languages or music) they may befamiliar with; real-time AR/MR rendering for facilitating visually orhearing impaired people to discover local surroundings' status withimages or sounds they may be able to see or hear; real-time AR/MRrendering and placement of native advertisements, etc.

FIGS. 3A-3C depict portions of a functional block diagram illustrativeof various components, blocks, network elements and/or apparatuses thatmay be combined into an example implementation or arrangement 300A-300Cof the network architecture of FIG. 1 according to one embodiment. Asensing point platform or system 303 shown in portion 300A including acollection agent (CA) 302 and a plurality of sensor systems such ascamera 304, GPS 306, accelerometer, sound, light, etc., collectivelyshown at reference numeral 308, is operative with an application supportlibrary 310 that can run a client data service application for purposesof scanning and uploading of sensory data and receiving/rendering ofoutput data. Where AR/MR technologies are involved, an AR renderingmodule 309 may also be provided. As noted previously, CA 302 may beconfigured to run on a suitable device platform (e.g., a mobile device)and operative to collect data from sensor systems 304, 306, 308 (whereprovided). CA 302 may also be configured to turn the sensors on or off,depending on deployment needs, such that no data may be collected whenthe sensors are turned off. In one arrangement, CA 302 may communicatewith an edge network node having an EDP platform, e.g., edge computelocation 318 shown in portion 300B, preferably via a RAN 314. Althoughother access networks or interfaces 316 (e.g., WiFi) may be used whereapplicable, it will be realized that some embodiments of the presentinvention can be particularly more advantageous when using RAN. Further,when communicating through a RAN, a suitable network data interface 326may be used in an example embodiment for collecting information aboutthe state of the network as will be set forth below in further detail.When CA 302 becomes available or comes online (e.g., car ignition in avehicular use case scenario), CA 302 connects to a Client Data Service(CDS) 320 hosted at the edge compute location 318. In one embodiment,CDS 320 may be configured to check a current data representation, ifavailable. If the current data representation is stale (e.g., beyond acertain age, which may vary by resource type), and unique (e.g., noother CA is providing this data), CDS 320 may be configured to instructCA 302 to provide the necessary data types, e.g., sensory data. In caseof RAN access, in an additional or alternative arrangement, CDS 320 maybe configured to further prioritize the need for new data against thecurrent network load, and only use paid—or higher priced data—if theinformation is relevant and/or urgently needed. In one exampleimplementation, relevance and/or urgency of the information may beevaluated and/or determined using statistical/probability analysis,heuristics, adaptive predictive modeling, etc. For example, a relativescore of “0” may be provided to indicate no or low relevance and/orurgency whereas a relative score of “10” may indicate a high degree ofrelevance and/or urgency. Responsive to prioritization (e.g., by CA 302,CDS 320, or in any combination), the sensory data may be streamed by CA302 to a local data store buffer 322 in the edge compute location 318.In one example embodiment, local data store buffer 322 may be configuredto accept full resolution data. In another variation, when possible, theraw data from sensing point 303 may be processed together with previousdata to update the local maps (i.e., world representations). In oneembodiment, local data processing 324 may be configured to provide orgenerate local world representations that may comprise a Point CloudRepresentation 332 (e.g., as points in a 2D/3D Cartesian coordinatesystem, polar coordinate system, etc.) of the sensed/scanned environmentor world, a wireframe representation 330 (e.g., triangles, etc., whichis a simplified model useful for rendering), and a texture datarepository 328. In one variation, world representation data comprisingpoint clouds 332, wireframe models 330, and/or texture repositories 328may be periodically synchronized with a central cloud compute location352 (shown in portion 300C), e.g., via a replication service 334 andassociated data service 336. The central cloud compute location 352 canalso request any of the data needed, e.g., via cloud-based replicationservice 356 and associated data service 362, wherein the requested datacan vary from a single image to a full video clip and time- andlocation-stamped sensor readings as needed. By way of example,cloud-based wireframe representation 358 and cloud-based texture datarepository 360 are illustrative of world representations hosted at thecloud compute location 352.

When a client needs the data (e.g., sensing point 303 also operating asa consumption point), for example, to render the data in an ARpresentation, it can request the data either from a central service(e.g., downloading an “offline mode” map for an area that may have poorcoverage) or, on demand, from central cloud locations 352 or edgelocations 318 as needed. In the example embodiment, this data may berepresented as at least the wireframe representations 330/358, andpossibly the texture data 328/360. The wireframe representation(s)330/358 allow(s) the AR rendering module 309 to identify where in the“virtual” world to render images such that they correctly overlay withthe physical world (e.g., as seen by a human driver in her field ofview). In one arrangement, such identification and placement ofAR-rendered objects in a physical field of view may be accomplishedusing the techniques set forth in the following commonly-assignedco-pending U.S. patent application: (i) “SYSTEM, METHOD AND ARCHITECTUREFOR REAL-TIME NATIVE ADVERTISEMENT PLACEMENT IN AN AUGMENTED/MIXEDREALITY (AR/MR) ENVIRONMENT”, application Ser. No. 15/609,683, filed May31, 2017, in the name(s) of Thao HT Nguyen and Meral Shirazipour,incorporated by reference herein.

It should be appreciated that providing output data in a pre-processeddata format means there is very little effort on part of the deviceneeded to accomplish data rendition. Additionally, the rendered datacould be furthered enhanced with third-party data sources (e.g., notdirectly from local sensors), and could be further formatted as to bepersonalized for the end consumption point (e.g., based on end user'sprofile data). Furthermore, the foregoing mechanisms described above canbe applied in variations of suitable combinations and sub-combinationsas well. For example, in a vehicular use case scenario, even if there isno RAN congestion, the system may elect to collect low-resolution datafor areas where there has been no change the past few scans, and, shouldit detect a change, have the next vehicle collect the higher-resolutiondata (e.g., as a data probe vehicle). It may also be useful in someexample embodiments to collect accelerometer data, position data (e.g.,GPS) and/or sound data, etc. alone without scanning visual data.

FIGS. 4A and 4B depict portions of a flowchart of various blocks, stepsand/or acts that may be (re)combined in one or more arrangements, withor without additional flowcharts of the present disclosure, forfacilitating optimized use of network capacity according to anembodiment of the present patent application. An example processcomprising portions 400A/B, which may be effectuated by way of a networkdata interface such as block 326 of the arrangement 300A-300C describedabove in respect of FIGS. 3A-3C, may commence when a collection agentcomes online (block 402) and connects to a client data service or CDS(block 404). Suitable geolocation data, e.g., GPS data,longitude/latitude data, postal address data, etc., may be obtained orotherwise determined by the collection agent, which may be reported tothe CDS, as set forth at block 406. A determination may be made at block408 as to whether environmental sensory data from the location of thecollection agent is needed, e.g., depending on whether the data is new,complementary to existing data, relates to a traffic observed in thevicinity of the associated sensing point, etc. If the sensory data fromthe reported location is not needed for some reason, the collectionagent may transition to a sleep or standby mode and/or may awaitcommands or instructions from the CDS (block 412) until new/additionallocation data is collected (block 406). Otherwise, if the sensory datafrom the reported location is determined as necessary, a furtherdetermination is made regarding whether the connection agent isconnected via a suitable RAN (block 410). Various further determinationsmay also be performed in association with RAN connectivity in additionalor alternative arrangements. For example, in a dual-mode or multi-modemobile device having the capability to connect to multiple cellularnetworks, possibly using different RAN technologies, relative signalstrengths, signal-to-noise (S/N) ratios, interference, and other channelindicators may be used in determining, estimating and/or selectingavailable radio network resources with respect to the reportingcollection agent. If suitable RAN resources are available, sensory datamay be transmitted between the collection agent and associated CDS(which may be associated with an edge compute location and/or acloud-based location), and may be mediated by way of push, pull,push/pull, or other transfer mechanisms (block 431). Further, as setforth at block 430, sensory data transfer or upload process may continueto take place in a loop until the collection agent is shut down, movesout of a radio service area, or upon instruction from CDS, etc.

If it is determined that the collection agent is not connected with aRAN (block 410), a further determination may be made as to whether a lowcost capacity is available, as set forth at block 414. If so, the lowcost capacity may be utilized in transmitting the sensory data asindicated at flow control point 426. If no low cost capacity isavailable for data transmission, a still further determination may bemade whether the data from the reported location is urgently needed, setforth at block 416 in portion 400B. As noted previously, determinationsas to data relevance, urgency, significance, reliability, etc. may bemade using a variety of statistical/probabilistic analysis, heuristics,AI, expert systems, adaptive predictive analysis, etc. If the sensorydata from the reported location is urgently needed, a still furtherdetermination may be made whether data at a lower resolution or coarsergranularity is acceptable, e.g., for the current time being (block 422).If so, the sensory data may be processed accordingly and transmitted asindicated at flow control point 426 via a low cost option for instance.Also, if the data from the reported location is not urgently needed, thedata may be locally stored (block 418). Likewise, if low resolution datais not needed for now, such low resolution data may be generated (block424) and stored as well (block 418). In one arrangement, the stored datamay be transmitted when network capacity becomes available, e.g.,responsive to a scheduling mechanism (block 420).

Skilled artisans will recognize upon reference hereto that a keyadvantage of the foregoing embodiment is the ability to collect andprocess a vast amount of data needed for AR/MR/VR world modeling, forexample, in a distributed and low cost manner, using knowledge of thenetwork state to update the virtual world in the background during lowcontention periods and only for matters of high priority (e.g., safetyapplications) during contention/congestion periods. In anetwork-assisted scanning mechanism as set forth above, an embodiment ofthe present invention therefore involves signaling the collection agentabout if any data is needed, what resolution is needed, whether to storedata and defer transmission, and performing such tasks/determinationsbased on balancing current network conditions versus the priority of thedata.

As discussed elsewhere in the present patent application, the HD mappingof surrounding environments is needed not only by autonomous/manualvehicle use cases but in other scenarios as well. By way of furtherexample, one demanding use case scenario is the real-time rendering ofthe environment for real-time AR/MR/VR content generation. Anotherexample is the use of AR/MR to enhance safety for drivers, whichrequires not only a precise mapping of the road environment but also atimely update of the environment. Such embodiments may be configured to,e.g., highlight precise locations on the road ahead (in AR/MR) that maybe dangerous above certain speed (e.g., AR/MR used to highlight blackice or a wide/deep pothole on the road). Accordingly, it should befurther appreciated that although the embodiments set forth in FIGS. 3and 4 highlight vehicular-related sensory data use case, the teachingstherein are equally applicable, mutatis mutandis, in other use casescenarios as well, e.g., a pedestrian's mobile device, smart wearables,smart infrastructure elements, AR/MR-based UE devices, etc.

Turning to FIG. 5, depicted therein is an example use caseimplementation within the network architecture of FIG. 1 thatillustrates one or more sensing points and one or more consumptionpoints according to an embodiment of the present patent application forpurposes of geospatial sensing of an area or region 500. An example roadsegment 502 disposed in the area 500 is comprised of a plurality oflanes 504A, 504B that may be traversed by a plurality ofautonomous/manual vehicles 510-1 to 510-N. As illustrated, a pluralityof obstacles 516-1 to 516-K are representative of obstacles such ase.g., bumps, potholes, ice patches, etc. that may be present in the roadsegment 502. Pedestrians 506 having mobile communications devices 508,pedestrians 512 having smart wearables 514 and smart infrastructureelements 516 as well as vehicles 510-1 to 510-N are operative withrespective collection agents and radio connectivity such that they mayoperate as environment sensing points with respect to respectivevicinities, wherein at least some of which may operate as consumptionpoints as well (for example, where a 3D model of the surroundingenvironment and/or vicinities including the road segment 502 may berequired for an AR/MR application). A plurality of RAN infrastructureelements 520-1 to 520-M are operative to effectuate suitable radio linkswith the sensing/consumption points using applicable technologies,standards, protocols, as noted previously. One or more inter-vehicularcommunications may be effectuated via V2V network paths 554 and 556 withrespect to the vehicles 510-1 to 510-K. One or more RAN edge elements520-1, 520-2 may communicate via backhaul/backbone networks 570, 572with a cloud-based datacenter (DC) platform 550 configured to hostappropriate CDS servers, data analytics, HD mapping databases, etc. Byway of illustration, RAN edge elements 520-1, 520-K may also communicatewith each other via appropriate network paths 574.

FIG. 6 is an example wireless network environment for facilitatingresource-based environmental data collection and processing in avehicular use case scenario according to an embodiment of the presentinvention. By way of illustration, example wireless environment 600 isshown as comprising a plurality of coverage areas, e.g., areas 604-1 to604-3, effectuated by appropriate wireless network infrastructureelements, e.g., base stations 602-1 to 602-3, corresponding thereto. Asis known in the art, a radio coverage area by a base station may takeany shape and include varying levels of signal quality and strength,wherein there may be some coverage overlap/gap depending on a number offactors. Illustratively, coverage areas 604-1 to 604-3 are exemplifiedas circular regions, each having a plurality of concentric signalquality/strength contours. By way of example, coverage area 604-3 isshown to include three such contours 606-1 to 606-3, each depicted witha respective signal level indicator icon, 612-1 to 612-3, respectively.Contour 606-1 having the highest signal quality is closest to basestation 602-3, whereas contours 606-2 and 606-3 are spaced further awayfrom the base station, with progressively deteriorating signal quality(i.e., strength, level, more interference, etc.). A plurality ofvehicles 610-1 to 610-N are illustrated as traveling along respectiveroutes 601 that may traverse various portions of the coverage areas604-1 to 604-3, wherein the routes may have varying obstacles, hazards,traffic congestion conditions, road surfaces, lane closures, etc., whichare generically shown at reference numerals 608-1 to 608-P. As anexample, UE-carrying vehicle 610-3 is operative as a sensing/consumptionpoint that is illustratively shown as traversing from coverage area604-3 to coverage 604-2, via a path or route 601 that involves a hazard608-1. Skilled artisans will recognize that whereas some vehicles have aUE that executes a collection agent application, other vehicles may beprovided with integrated vehicle controller units having the collectionagent functionality for purposes of the present invention.

To facilitate network-assisted scanning, a number of network nodes orelements may be configured to interface with or otherwise be deployed aspart of the wireless network environment 600. For example, amobile-optimized edge network node 614, which may be provided as part ofa radio network controller (RNC) node serving base stations 602-1 to602-3, may be configured to host a CDS application as discussed above indetail. A data analytics platform 618 may be co-located with orotherwise coupled to the edge network node 614 for local data processing(i.e., edge compute location). A local storage of processed datasuitable for accelerated AR/VR/MR processing/rendering as well as HDmapping may be provided as a database 620 wherein the data may beindexed to the geolocation data of the sensing points depending on theirreporting locations and/or collated, correlated, or otherwise combinedwith third-party data services 622. Also, in one embodiment, the edgenetwork node 614 may be interfaced via the Internet 624 to a datacenter626 and associated cloud-centric database 628.

As the sensing points 610-N traverse through the coverage areas 604-1 to604-3 via respective routes 610, their signal qualities with respect tothe serving base stations may vary, including potential service outage.Accordingly, signal quality of different sensing points with respect tothe same obstacles may also vary, some having better quality thanothers, depending on which base station is used. CDS applicationexecuting at the edge network node 614 may therefore select data fromdifferent sensing points so as to obtain best quality data fromappropriate sensing points with respect to the surrounding environmentincluding traffic/road conditions and obstacles 608-1 to 608-P. In otherwords, if two sensing points are capable of reporting sensory data withrespect to a particular obstacle/condition, one connected in a betterRAN coverage area than the other, the CDS service logic may beconfigured to select the sensory data from the better-connected sensingpoint with respect to the sensed obstacle/condition. As notedpreviously, where the data transfer is being deferred, collected datamay be cached and scheduled for delivery according to a suitablescheduling mechanism. Skilled artisans will therefore recognize thatvarious combinations of sensing points, signal strengths, channelconditions, transport costs, data relevance/urgency, etc. may beimplemented for optimally obtaining the sensory data pertaining to thesurrounding environment, preferably at varying resolutions, in anexample embodiment of the present invention.

Among the several advantages of the present invention, it will befurther realized that an example embodiment provides for placing therelevant data at a nearest edge computation node, wherein localizedgeospatial data may be leveraged by performing initial processing of theraw data and turning it to a simplified model in a local processingnode. Typically, this arrangement poses little to no detriment tooverall data quality and, accordingly, an example embodiment of thepresent invention may be configured to send far less data upstream forcentral processing, thereby reducing overall bandwidth usage. Skilledartisans will further appreciate that this practice can also beadvantageous with respect to privacy concerns depending on how datacollected is being pre-processed and encoded. Since data retrievaland/or scheduling may be coupled with prioritization, the localprocessing can reduce costs further by being intelligent about how andwhen it receives data. Depending on traffic density and network load, itcan increase or decrease the frequency or resolution of the data thatclients may upload, or signal them to defer it to a later cell towerthat may have more capacity. As network usage may shift drasticallyquickly, e.g., depending on routes being taken, dynamic/unpredictabletraffic conditions, etc., a local processing node is better suited forthese types of decisions.

Moreover, by applying priority principles and policies, an embodiment ofthe present invention may be configured to further lower the cost ofdata transfer by selectively choosing when and where to transmit data asnoted above. Armed with the knowledge of current network usage,encounters of a significant road anomaly, such as a fallen tree, anaccident, a sinkhole, flooding, debris, or any number of anomalies(whether actual or fictional, e.g., mirage) in an environment may beappropriately handled. For example, an initial scan of the area canlikely be of low resolution; once processed, the cloud service canrecognize the presence of an anomaly and request the nextvehicle/sensing point in the area to perform a high-resolution scan.However, especially in the case of an accident, the network may becongested because of it; emergency vehicles, autonomous UAVs or AUAVs,news crews, or even passerby humans using social media can place a highload on the network. An embodiment of the present invention cantherefore be advantageously configured to obtain knowledge of the loadon a nearby cell or coverage area and determine that it can receive thedata within a certain amount of time (e.g., 45 seconds later) due to thetravel time to the next cell, but minus the transfer time gained fromthe additional bandwidth, potentially at a lower cost, as part ofoptimization balancing of cost and data delivery as previously noted.

Furthermore, as an embodiment of the present invention can be configuredto provide sensory data updates at a fixed or lower cost, in addition toutilizing unused network capacity, e.g., on a scheduled basis, toincrease the resolution of the scans. For instance, an embodiment of thepresent invention can be configured to free up network capacity ifneeded, and detect there are no significant updates needed at currenttime. Additionally or alternatively, the system can adjust, throttle orchange the resolution of scans based on the knowledge of the edgenetwork in a dynamic, on-demand manner. One reason for this is theability to locally monitor and control data in manageable chunks, wherethe quantity might instead become too bulky for central processing. Forexample, consider the following traffic condition scenario. On anaverage day, X vehicles per minute may pass by a location, increasing to2X or 3X vehicles per minute during peak times. This volume can easilybe managed in-memory by a local edge processing node, but may requirecomplex caching and tiering operations for full central processing(e.g., a cloud).

In a still further arrangement, an example embodiment of the presentinvention may be configured to provide increased spatial resolution,including at varying levels of granularity. For example, instead ofuploading the road data only at specific road segments, or ignoringaccelerometer data under a certain threshold, or recording visual dataonly at certain points (e.g., a street view only being available atintervals), a network-assisted processing mechanism at a localprocessing center may be configured to instruct the scanning device orsensory system to upload a higher resolution dataset on an area thateither has not been scanned before, or needs additional updates only. Ascan be appreciated, sending the information to a consumption point suchas a moving vehicle as rapidly as possible is of particular importancesince the system, for privacy or security reasons, may not be informedwhether the selected vehicle will stay at the current reporting location(e.g., an intersection) or move to another one.

It will be apparent to skilled artisans upon reference hereto that abase system according to the foregoing teachings may be utilized in avehicular use case scenario with particularly advantageous featureswhere detailed road data may be provided for facilitating precisedriving recommendations, commands or advisory messages in a furtherimplementation. Set forth below are further aspects that describeadditional embodiments with respect to providing such highly assisteddriving (HAD) or advanced driver assistance system (ADAS) control forpurposes of the present patent application.

Broadly, an overall example system with respect to a vehicularnavigation recommendation use case is set forth immediately below, withadditional details and embodiments being provided in later sections. Anexample arrangement of the present invention may be configured, as partof the course of navigation of a vehicle, to perform a simulation of theroute, and suggest changes to the route, such as using a different laneof travel, using a better road nearby, or even which part(s) of the laneto utilize to skip specific obstacles (e.g., the most egregious bumps).This information may be provided for use by a human driver, or fed intoan autonomous vehicle to facilitate a routing path (e.g., undersupervisory control) such that the passengers get the smoothest ride andcan enjoy whatever other activity they may be engaged in. Even withrespect to a human driver, the data may be rendered and/or utilized innumerous ways. For example, a traditional GPS can consume the data andprovide an output by either recommending a different, better route, orby using existing lane assist facilities to suggest to the driver thatchoosing a different lane, at least for a certain duration, can providea smoother ride experience, or by suggesting a speed to travel for theaffected stretch, or possibly combining the aforementioned. Also, a newvisual representation may be provided that uses the spatial/positioningdata gathered to overlay an AR “racing line” for the driver, showing notonly which lane(s) to use, what speed(s) to use, but also which part(s)of the lane to use to get the ideal driving experience.

In one arrangement, an example embodiment may be implemented as a systemcomprising two components or subsystems, which may overlap in parts. Afirst component, portion or subsystem relates to data collection andprocessing, which is responsible for building a suitable knowledgebaseof the world. As noted previously, this portion may be implemented usingan embodiment of the network-assisted scanning of surroundingenvironments described in detail hereinabove although other dataacquisition and processing mechanisms may also be used in an alternativearrangement. However, providing a network-assisted scanning mechanismaccording to the teachings of the present patent disclosure canadvantageously allow for low-cost data gathering and navigationalassistance processing, and thereby making it possible to overlay thedata in enhanced, user-friendly presentations using AR/MR.

The second component, portion or subsystem involves the client-sideapplication of providing navigational guidance and AR display to a humandriver, or a 3D model of expected path routing for an autonomousvehicle, or a combination thereof. In either the autonomous vehicle caseor manual vehicle case, a collection agent may be provided as part of amobile device that may be mounted to the vehicle (e.g., referred to asan in-vehicle unit) or the collection agent functionality may beintegrated within the vehicle for operating in conjunction with theon-board sensory devices, vehicle systems, diagnostic systems, displayunits, and the like, albeit at a higher initial investment cost.

In one arrangement, a base system embodying network-assisted geospatialscanning may be implemented for collecting road smoothness data. It willbe appreciated that the general methodology for data collection may beconfigured to follow roughly the same process flow as described before,mutatis mutandis, except that the urgent data collection mode may not beused all that frequently. The collection agent may be configured in oneembodiment to collect audio data indicative of road irregularity (e.g.,conditional on user approval) for improved recognition of roadsurfacing. While the urgent data collection is no longer needed, theedge compute node may still be used for all other benefits, however.After the data is collected, processing for AR and HD world mapping canoccur. Additionally, other sensory data relating to the vehicle, e.g.,GPS data, optical image data, accelerometer, inclinometer, vehicleperformance (e.g., ABS, steering, suspension, traction control, speed,etc. if available) and sound data may also be processed in order toidentify the presence of road irregularities, their magnitude, as wellas their location. This data may be used as raw input data in thefollowing ways in an exemplary embodiment. In a first process flow, thevehicular usage data is calibrated to determine, e.g., vehicle type,class, etc. (i.e., calibration flow). Calibration may occur during thebeginning, middle, or end of a driving session, depending on where thevehicle has an overlap with existing road data. It should be appreciatedthat since the reporting devices, e.g., mobile phones, whose placement,or even vehicle, may change from session to session, it is important toperform self-calibration in such usage conditions. Also, calibration canbe done against other vehicles that have driven the same road (e.g.,used as probe vehicles or sensing points). If that cannot be performed,e.g., due to lack of overlapping road/lane positions, the data may bestored and used for calibration later, e.g., to detect if the user isdriving the same vehicle.

In one configuration, an example calibration flow may primarily focus onidentifying a vehicle's response to road irregularities. This responsedepends on, inter alia, the reporting device's mounting position, thehardware used to attach to device (e.g., if a cell phone is used), theinflation, tread and type of a vehicle's tires, as well as thesuspension and chassis characteristics. If the deployment involvesbuilt-in functionality integrated within the vehicle, some of theseparameters may be determined by reading a controller area network (CAN)bus provided in the vehicle (e.g., tire inflation), or may bepredetermined (e.g., mounting location). Lack of such information maymean calibration merely attempts to classify the vehicle at more genericlevel, e.g. “mounting position 4 of a sports car”. In one arrangement,lane data (e.g., obtained from lane indicators or sensors) may used toidentify the width of the car, while road irregularity data (e.g.,impacts from driving over a crack or a rail) can be used to estimate thelength of the wheelbase.

In a second process flow, the data may be used to determine which road,direction and lane the vehicle is travelling in (e.g., navigation data).When an irregularity is detected, e.g., primarily from accelerometer andaudio data, the AR position derived from the base system may berecorded. This position can then be used to cross-reference the texturedata from the given area to attempt to identify the type ofirregularity, its exact location and size, its trajectory (if moving),and the like. Responsive to the two process flows above, an embodimentof the present invention can be configured to update the HD mapping andnavigational-assist database, depending on the granularity of inputdata, if possible.

To further concretize at least a portion of the foregoing principles,attention is now directed to FIG. 9, wherein a diagram showing arepresentative manual/autonomous vehicle 900 is depicted as havingvarious sensors and on-board monitoring devices operative with acollection agent integrated within the vehicle or in conjunction with aUE device placed within the vehicle for purposes of an embodiment of thepresent patent disclosure. By way of further illustration, vehicle 900may be provided with a vehicle controller unit 912 that can beconfigured to operate as a sensing point or collection agent forobtaining ride quality and other vehicular data, which vehiclecontroller unit 912 may be coupled to various sensors and vehiclesystems via a CAN. Where a wireless UE, e.g., cell phone, is used as acollection agent, the wireless UE may be mounted to or placed within thevehicle 900 at a specific location. Optionally or alternatively, where abuilt-in CA arrangement is provided, appropriate connectivity to the CANbus may be achieved for obtaining various pieces of vehicular parametricdata. A suspension damper system 924 may be coupled to the controller912, which is further coupled to one or more suspension sensors 914 andone or more wheel sensors 916. A man/machine interface (MMI) 926 may beprovided as part of a dashboard that allows a driver (i.e., in manualvehicle configuration) to interact with various vehicle controls,including but not limited to adjusting the suspension, monitoringperformance, selecting drive mode (e.g., sport, comfort, etc.).Controller 912 may also be adapted to communicate with a modem or otherwireless communication module 920 for transmitting ride quality data(where locally obtained) to a remote server via an antenna 922. In onearrangement, controller 912 may be configured to continuouslycharacterize ride quality as vehicle 900 traverses over a plurality ofroad segments based on signals from sensors 914, 916 as well as othersensed variables. In order to index the sensed data with geographiccoordinates so that the ride quality data can be associated withparticular pre-defined road segments, a location device, e.g., GPS 918,may be coupled to the wireless communication module 920 and/orcontroller 912.

FIGS. 10 and 11 depict block diagrams of components of a vehicle havinga collection agent for facilitating HAD/ADAS according an embodiment ofthe present patent disclosure. Vehicle 1000 shown in FIG. 10 is arepresentative block diagram of vehicle 900 described above, whereinvehicle control unit and/or collection agent 1002 is operatively coupledvia a controller bus 1022 to various vehicular systems such assuspension system 1032, anti-lock braking (ABS) system 1034, steeringsystem 1036, acceleration system 1038 and other vehicular systems 1040.By way of illustration, other vehicular systems 1040 may comprise,and/or be provided as part of, at least one of: adaptive cruise control(ACC) system, automotive navigation system, blind spot monitoringsystem, collision avoidance system, crosswind stabilization system,human driver monitoring system, forward collision warning system,intersection assistance system, hill descent control system, tractioncontrol system, axle sensor system, lane departure warning and lanechange assistance system, driving mode system, proximity/parking andobstacle detection system, tire pressure monitoring system, traffic signand image recognition system, and turning assistance system of thevehicle, and the like.

FIG. 11 depicts additional components, modules, blocks and/or subsystemsthat may be provided as a vehicle control unit 1102 in a specificembodiment of the vehicle control units 912, 1002 set forth above. Asillustrated, vehicle control unit 1102 may include a processor 1104,memory 1106, radio 1102, camera(s) 1114, and sensors 1116. Memory 1106of vehicle control unit 1102 is operative to store informationaccessible by processor 1104, including program instructions 1108 thatmay be executed by the processor 1104. Memory 1106 may also include data1110 that may be retrieved, manipulated or stored by the processor 1104.As noted previously, the memory may be of any type of tangible mediacapable of storing information accessible by the processor, such as ahard-drive, memory card, ROM, RAM, DVD, CD-ROM, Flash, etc. Sensors 1116may include sensors for detecting pitch, yaw, or roll of vehicle 1102.Also, sensors 1116 may comprise a laser scanner, sonar, radar, Lidar, oranother similar device for detecting obstacles and/or range detection ingeneral. Camera(s) 1114 may be one or more cameras for taking picturesof the surroundings of vehicle 1102 (front view, rear view, side views,top view, 360-degree view, etc.), which may be used to detect roadirregularities, road blocks, traffic lanes, other vehicles, pedestrians,and other environmental information, as noted previously. Radio 1112 maycomprise a wireless device operative to effectuate short-range and/orlong-range radio communications using any known or heretofore unknownradio communications technology, for receiving and transmittinginformation. Radio 1112 may be connected to a server 1120 (e.g.,operative as an edge compute location) over a wireless network 1118.Broadly, it will be recognized that vehicle control unit and/orcollection agent 912/1002/1102 set forth above may be configured togenerate, sense, determine, or otherwise obtain various pieces ofvehicular condition data and vehicular navigation data from the variousvehicular systems in conjunction with the sensors described herein. Asdescribed in detail hereinabove with respect to the network-assistedscanning system embodiment of FIGS. 3A-3C, edge compute server 1120 maystore information that may be used by the vehicle control unit 1102,whether in autonomous mode or under human supervision. Such informationmay include HD maps, information about other vehicles' trajectories,road conditions, climate conditions, and so forth, which may bepresented in one or more AR/MR displays.

FIG. 12 depicts an example geographical region/area 1205 having a roadnetwork 1212, 1214, 1216 traversed by a plurality of manual/autonomousvehicles 1204, 1206, 1208 operative to collect environmental data inaccordance with the teachings of the present patent disclosure. Asillustrated, vehicle 1206 is traversing a road segment 1214 having endpoints at 1218 and 1220, for example. In one arrangement, vehicle 1206is operative to monitors signals from GPS satellite network 1210 todetermine the geographic position of vehicle 1206 and ascertain theidentity of road segment 1218. Similarly equipped vehicles 1204 and 1208traversing different road segments 1212 and 1216 may likewise gatherride quality data and other sensory data which may be indexed withrespective geographic coordinates also obtained by monitoring GPSsatellites 1210 and then remotely uploaded to a crowd-sourced datarepository (e.g., at respective edge node compute locations).

For providing communication with respect to vehicles 1204, 1206, 1208, awireless cellular network including one or more cell sites 1222 may beprovided in a manner similar to the arrangement shown in FIG. 6described hereinabove. A data network 1224 is operatively coupled to aserver platform 1230, which may include a supervisory agent 1226embodied as a CDS as well as a ride quality/obstacle database 1228. Aroute planning function 1232 utilizes the ride quality/obstacle datafrom server 1230 in order to identify the ride quality or comfort levelassociated with various potential routes or racing line paths availableto a vehicle, e.g., vehicle 1206, between the endpoints 1218, 1220 ofthe road segment 1214. Whereas the route planning function 1232 may beintegrated or otherwise associated within as part of server 1230, it mayalso be located on-board an individual vehicle (e.g., thick clientapplication), or performed by other servers or resources as part of anavigation service provided to subscribing drivers, for example.

Turning to FIGS. 13A and 13B, depicted therein are portions of afunctional block diagram illustrative of various components, modulesblocks, network elements and/or apparatuses similar to the foregoingembodiments that may be configured as a system 1300A/B operative inassociation with the network-assisted scanning architecture of FIGS.3A-3C for facilitating highly assisted driving based on up-to-dateenvironmental data according to one arrangement. A vehicle 1302,exemplary of vehicles 900, 1000, 1204-1208 described above, is providedwith a data collection point, e.g., a cell phone, operative as ameasuring device 1304, as illustrated in portion 1300A. Sensors/systemssuch as camera 1306, accelerometer 1308, video 1310 and GPS 1312, may beassociated with the measuring device 1304.

Additionally, optionally and/or alternatively, the cell phone inputs canbe substituted with optional monitoring devices (OMDs) built into thecar at various locations, e.g., OMD 1320 associated withchassis/suspension 1318, OMD 1321 associated with a front axle 1322, OMD1323 associated with a rear axle 1324, and the like. Generally, themeasuring device 1304 may use a fixed mount 1314 to the vehicle 1302,e.g., at dashboard 1316 or windshield, positioned so that it can observethe road ahead as well as the surrounding vicinity depending on therange of the sensory equipment. It will be seen that suchplacement/layout of sensors and/or measuring device can simplify anexample calibration classifier process, responsive to various pieces ofvehicular condition data and/or navigation data, as will be described indetail below.

Data transfer may be effectuated via a network 1328, whereby a basesystem as described above may process the sensory/environmental data todetermine the vehicle's AR position with respect to a road segmentand/or surrounding vicinity. A simplified example base system 1333 isillustratively shown in portion 1300B as comprising a CDS/database 1330as well as AR-facilitating data e.g., wireframe/texture output 1332.Once the vehicular AR position is determined, the detail of which issignificantly higher than GPS alone, the vehicle's direction andlocation can be determined on the lane or even at within-lane level, aspart of a vehicle position classifier block 1350. In one arrangement,vehicle position classifier block 1350 may also interoperate with orotherwise include in-lane position block 1352, lane selection block1354, and road location and direction block 1356. This data can then beused to determine the vehicle's wheelbase. Further, the incidence ofencountering localized road conditions (e.g., lane width indicators,well-known irregularities, bumps, etc.) may be utilized to infer whenone side of the vehicle has encountered the irregularity, giving a widthestimate. Likewise, the time between two spikes due to bumps on the roadmay be utilized to estimate the length of the wheelbase, e.g., dependingon sensor locations on the axles, suspension dampers, etc. It should beappreciated that in one configuration, many samples/measurements may betaken and adaptively processed (e.g., in conjunction with knowndatabases) to obtain highly reliable inferences. For instance, thevehicle type can be inferred by comparing the impact of the roadirregularities compared to other known vehicles. By way of illustration,suppose a given pothole is identified, has a known position, and has anormalized accelerometer reading of 5 (an arbitrary value). A sports carhitting that pothole may read a higher value (e.g., a reading of 8)because of the tighter suspension, while a family SUV might only be avalue of 3 (i.e., a lower reading). Identifying the vehicle typetherefore allows for comparison between readings from differentvehicles, which can provide a rough indication of the suspension type,which in turn is an indicator of the vehicle size, weight and type. Anexample calibration classifier block 1334 is therefore operativelyassociated with vehicle type determination block 1336, wheelbaseestimation block 1338 and sensor position block 1340.

Additionally, as noted above, the vehicle's position is also classifiedor determined in conjunction with the AR data by vehicle positionclassifier 1350. At the lowest fidelity source, the system can provideat a minimum the road location and direction. However, by using ARpositioning data, an example embodiment of the system 1300A/B can beconfigured to deduce not only which lane but also what part of the lanethe recording/reporting vehicle is traveling in. The two datasources—vehicle type and position—output by calibration classifier 1334and vehicle position classifier 1350 can then be used to normalize thedata into a plurality of output buckets for further analysis. In oneexample arrangement, one data output may comprise or relate to thedetermination of the obstacle size and position. A second data outputmay comprise or relate to a probability distribution for the obstacle'slocation in the AR world surrounding the vehicle. A third data outputmay comprise or relate to a depth map determination of the expecteddisplacement (e.g., depth/height or other contours) of the irregularity.In a further embodiment, the system 1300A/B may be configured tocalculate or contribute to a lane-specific International Roughness Index(IRI), a standardized measurement of the roughness of a stretch of road,including parts thereof, as well as a general road IRI. As exemplifiedin FIG. 13B, an output data block 1360 of the system portion 1300Btherefore is illustratively interfaced with obstacle size determination1362, lane IRI determination 1364 and road IRI determination 1366.

According to the teachings of the present patent application, a routeplanning module 1368 (e.g., roughly analogous to the module 1232 shownin FIG. 12 described above) may be configured to receive the foregoingdata outputs (i.e., pertaining to road smoothness/roughness, ridequality, etc.) for further analysis with respect to determining anoptimal racing line path for the vehicle 1302. Depending on what/who theconsumption point is and/or what the data is used for, whethersubscription-based or not, etc., different modalities may be implementedin an example arrangement. For general route planning, module 1368 canbe configured to calculate, determine and or otherwise estimate the“smoothest” route taking into account vehicle length. It should beappreciated that this determination would be particularly advantageous,for example, for fleet planning when options are available to reduce thevehicle operating cost. For human-based navigation, route planningmodule 1368 may be configured to provide lane recommendations (e.g.,“stay in this lane for X miles, then switch to the one to the right”),where it is legal to do so. The system may also be configured to providerecommendations on which parts of the lane may be smoother, when it maybe a good idea to slow down to avoid the impact of a larger bump, and,for slower or country roads, how to navigate the road to avoid large andpotentially dangerous obstacles. For autonomous driving mode, the systemmay be configured to provide a high fidelity topographical map (e.g.,accurate to within inches) in addition to the road map, allowing thevehicle to perform various self-smoothing operations, e.g., adjustsuspension, take corrective action (e.g., if possible withoutinterfering with other vehicles) or adjust speed, etc. Moreover, the mapdata can also be used to make predictive adjustments on in-vehicledisplays, e.g., correct for a road bump in an AR display.

FIG. 16 is a flowchart of various blocks, steps and/or acts that may be(re)combined in one or more arrangements, with or without additionalflowcharts of the present disclosure, for facilitating a process 1600operative in association with the system of FIGS. 13A-13B according toan embodiment of the present invention. At block 1602, data is collectedand/or received by a vehicle collection agent, and/or in associationwith monitoring devices and sensors, if any. The collected/received datamay be transmitted to a CDS, e.g., either immediately or at somescheduled point. An embodiment of a base system as described in detailhereinabove computes, calculates or otherwise determines AR positioningdata in suitable data formats (block 1604). The vehicle type may beclassified, calibrated, either based on vehicle known data (e.g., fromon-board sensors) and/or based on the handling of the vehicle andestimated vehicle parameters (e.g., wheelbase, etc.), as set forth atblock 1606. Road lanes, road conditions, vehicle lane position anddirection are determined or otherwise obtained at block 1608. Using theforegoing pieces of data, a calculation or determination is made forobtaining a fairly accurate vehicle location, velocity (e.g., differencebetween two locations), etc., which may be normalized based on thevehicle class/type determination (block 1610). Normalized data may beused for updating obstacle probability maps, IRI and road conditiondata, etc. As noted above, an optimal racing line may also be computed,determined or otherwise estimated, e.g., for presentation inAR/VR/MR-friendly display (block 1612). Depending on the controlmodality, the vehicle's routing/motion may be adjusted responsive to theracing line characteristics, e.g., via autonomous control and/or manualoverride (block 1614).

Skilled artisans will recognize that it is not always necessary for thevehicle to send the data to a network node in an example embodiment ofthe foregoing process. For instance, two modes may be provided dependingon power management, e.g., to conserve power. First, thevehicle/collection agent can record but not send data until and unless asensed parameter's values (e.g., the accelerometer readings) exceedcertain thresholds. At that point, it can send a historical record thathelps find the irregularity. In another embodiment, a deeper powersaving mode (i.e., second mode) can avoid recording outside of locationswhere there is not a known and identified need for more data, and onlytrigger a notification on the identification of a road irregularity. Asthere is no recording of the area, this will only leave a marker on theserver to request that a later vehicle indeed scans this area (e.g., bya dedicated probe vehicle or other vehicle having the collection agentfunctionality).

FIG. 15 is illustrative of a vehicle 1500 with AR display capability forshowing a routing path or racing line presented based on up-to-dateenvironmental data according to an embodiment of the present invention.A windshield 1505 of vehicle 1500 offers a human driver a view of a road1502 on which the vehicle is traveling. A plurality of obstacles, e.g.,potholes, 1504, 1506, 1508, may sensed by a network-assisted scanningand processing system of the present invention (e.g., system 1300A/B ofFIGS. 13A-13B). Responsive thereto, and based on vehicle calibration andnavigation data, an optimized racing line 1510 may be presented in an ARfield of view 1503 virtually superimposed on or visible though thewindshield 1505. In one arrangement, the racing line 1510 may becomprised of different segments 1512, 1514, 1516, which may be ofdifferent thicknesses, different color-coded portions, and the like,that can be configured to indicate different vehicular performanceconditions and parameters, e.g., speed, suspension/bumpiness, etc.Although such a simulated AR racing line may be presented as a highlyassisted driving aid to help a driver safely navigate a road with unevensurfaces and obstacles, a driver (human or automaton) may or may not acton that information, depending on the road size and speed. Further, anexample embodiment may present more information or less informatione.g., graphics, in an AR view, depending on road/driver/climateconditions, so as to minimize distraction.

It should be appreciated that the terms “augmented reality” or “AR” and“mixed reality” or “MR” may be used somewhat interchangeably forpurposes of an embodiment of the present invention. Further, where only“AR” or “MR” is mentioned, it will be realized that these termsrepresent both AR and MR, cumulatively or otherwise. In the context ofthe present patent disclosure, augmented reality (AR) is a technologywhere the real world and its physical objects, images, senses, sounds,and other tangible quantities in a physical environment that is viewed,sensed, heard, or otherwise perceived by a user using a suitabledisplay/device and other related hardware is augmented or supplementedwith or by virtual objects or other computer-generated sensory inputsuch as sound, video, graphics, olfactory and tactile sensory data, aswell as suitable GPS/mapping data in ambient-assistedmobility/navigation cases. In a general sense, AR may be an overlay ofcontent on the real world, but that content may or may not be anchoredto or part of the physical view or its objects. On the other hand,virtual reality (VR) uses computer technology to create a simulatedenvironment in which the user/consumer/subscriber is completely immersedin the simulated experience. In a virtual reality environment, all thevisuals are digitally produced and there is typically no interactionwith the real world. More broadly, embodiments of the present inventionmay treat mixed reality (MR) as a mix of AR and VR, sometimes alsoreferred to as “hybrid reality” that involves a merging of real andvirtual worlds to produce new environments and visualizations wherephysical and computer-generated objects, sounds, images, etc.(collectively, “entities”) may coexist and even interact in real time.In other words, MR can be considered an overlay of synthetic entities orcontent on the real world environment that are anchored to and interactwith the physical objects/entities therein in some meaningful fashion.Thus, in an MR environment, an embodiment may not only allow for themerger of digital objects within a real world scenario but alsofacilitate extra real life textural, tactile, olfactory, visual, aural,or other sensory feedback such as “depth” or “surfaces” or “texture”,and the like. Turning to FIGS. 14A-1 to 14A-3, 14B and 14C, depictedtherein are flowcharts or portions thereof having various blocks, stepsand/or acts that may be (re)combined in one or more arrangements, withor without additional flowcharts of the present disclosure, forillustrating additional details with respect to the system of FIGS.13A-13B. One skilled in the art will recognize that processes 1400A-1 to1400A-3, 1400B and 1400C shown in FIGS. 14A-1 to 14A-3, 14B and 14C maybe executed at a network node in combination with a network-assistedscanning system and CDS (e.g., edge network and/or a cloud-based computelocation), although other arrangements are also possible, e.g., withdistributed control involving client-side functionality. Upon commencingas set forth at block 1402 of process portion 1400A-1 in FIG. 14A-1,which may be responsive to an input from a base system and/or datatransfer from a collection agent, for example, a determination may bemade whether calibration is needed (block 1404). If so, a furtheranalysis may be performed as to whether automatic calibration oruser-mediated calibration is required, as set forth at decision block1406. At block 1408, a user query process is engaged for savingcalibration data if different (block 1410). Under automatic calibrationflow, a determination is made whether the vehicle has on-boardcalibration (block 1412). If so, factory calibration data may be loaded(block 1414) and saved (block 1410). Otherwise, an iterative calibrationprocess is executed until a sufficient number of data samples have beenobtained (block 1416). As part of acquiring calibration data in aniterative process, shown in process portion 1400A-3 of FIG. 14A-3, inputdata relating to, e.g., accelerometer, timestamp, geolocation, etc. maybe fetched or otherwise received from a node, e.g., a base systemdatabase (block 1420). If a condition relating to an anomaly is detected(block 1422), it is identified and characterized in reference to thevehicle location and performance parameters (block 1426). For example,velocity at impact, anomaly vector compared to gravity minus noise,etc., may be determined. Subject to or conditional upon a thresholdevaluation, if a determination is made that an anomaly is not present inreference to the reported location (block 1430), the flow controlreturns to input data retrieval 1420 in an iterative loop. Likewise, ifan anomaly condition is not detected at block 1422, vehicular parametersrelating to velocity, aggregate accelerometer readings, gravity vector,etc. may be determined as per normal navigation (block 1424), whereuponthe flow control returns to input data retrieval block 1420 as well.

On the other hand, if an anomaly is determined to exist (decision block1430), anomaly data may be processed as set forth at block 1432, whichoutput may be provided to a number of sub-processes with respect tosample vehicle type, vehicle wheelbase sampling, sensor mountingposition sampling, as set forth at blocks 1434, 1436 and 1438,respectively. Thereafter, the flow control may return to decision block1416 in an iterative loop until the calibration process is complete.

After a sufficient amount of calibration data has been obtained, it maybe summarized and applied to any data samples gathered (block 1418). Anyremaining data samples may be uploaded (block 1419) and saved (block1410).

It should be appreciated that calibration can occur during many stages,as a background process or conditioned upon an event and/or user input,such as, e.g., after the accelerometer encounters an impossible event(e.g., gravity turning upside down). As a background calibrationprocess, the foregoing iterative loop involving blocks 1416-1438 may beimplemented in an example arrangement. As to a conditional calibrationcase in one scenario, this could be in response to the mobile phonepossibly being picked up by a user, dropped or moved, etc. It ispossible for a user to change (intentionally or not) the mountingposition of a phone, or even take it with him while changing vehicles,or replace the phones. A (re)calibration can therefore be executedsubstantially similar to the foregoing iterative process, as exemplifiedby blocks 1448, 1450, 1452, 1454 and 1456 of process portion 1400A-2.Accordingly, a calibration process may go back and forth between abackground calibration process and a conditioned (re)calibration processdepending on the situation.

Upon user selection 1440, the process flow may exit (block 1442), or mayselectively lead to one or more assisted-driving processes such as,e.g., steering guidance 1444, route guidance 1446, among others, similarto the embodiments as described previously. An example route guidanceprocess 1400B in further detail is set forth in FIG. 14B as anembodiment of block 1446 of portion 1400A-2 of FIG. 14A-2. Likewise, aflowchart with additional details relating to an example steeringguidance process 1400C is set forth in FIG. 14C as an embodiment ofblock 1444 of portion 1400A-2. Upon starting (block 1460), guidanceprocess 1400B involves obtaining destination address information, e.g.,street, postal, GPS. latitude/longitude, or other coordinates, etc.(block 1462), as well as a vehicle profile (block 1464), which areprovided to a server (block 1466). A search loop, which may typicallylast until either a configurable search time has been exceeded or adesired number of suitable routes have been found, may be initiated at ablock 1468. Initially, a new/potential candidate route is created,designated or identified (block 1472) for which expectedsmoothness/roughness/comfort level may be determined (block 1474). Inone arrangement, the smoothness profile may be adjusted according to thevehicle profile (block 1476). A determination may be made as to whetheran intended/desirable level of smoothness/comfort level has been met(block 1478). If not, the flow control loops back to the searchinitiation block 1468. On the other hand, if the candidate route has arequisite level of smoothness/comfort level as per decision block 1478,the route is prepared, including but not limited to lanerecommendations, surface irregularities, etc. (block 1480). Thereafter,the prepared route is added to a route list (block 1482), whereupon theprocess flow returns to the search initiation block 1468 for nextiteration. When the search loop is done or otherwise exited, the foundroutes may be provided to the vehicle (block 1470) via an interface aspart of display guidance relative to a map (block 1484). If no routeshave been found, a suitable message may be provided instead.

Turning to FIG. 14C, steering guidance process 1400C commencing at block1485 proceeds to obtain a vehicle profile (block 1486). A preciselocation of the vehicle is obtained (block 1487), e.g., from a basesystem as pointed out previously.

If more road data is needed, that information may be provided via asuitable mechanism (e.g., push mechanism), as set forth at block 1488. Abest lane may be calculated, obtained or otherwise determined (e.g., ifnot supplied by a route guidance process), as set forth at block 1489.If a lane change is needed (block 1490), an advisory message to switchlanes may be provided (block 1493). In one arrangement, in-lane assistservice may be disabled so as to reduce or avoid distracting the humandriver. If lane switch is not needed, a best path through the determinedlane is calculated or otherwise determined (block 1491), whereuponvelocity with respect to the determined best path may beadjusted/corrected so as to reduce perceived impact (block 1492). Aroute map and associated AR display view may be updated to show thecorrected best path to steer in the determined lane (block 1495). In onearrangement, output from lane change advisory message block 1493 and/ormap update/AR display block 1495 may be provided to the vehicle locationdetermination. In a further arrangement, “best path” determination maybe modulated based on driving styles, while continuing to apply a set ofconstraints to ensure reasonable steering in order to stay safely in thelane. One skilled in the art will recognize the foregoingguidance/steering processes may also combined into a combination HADprocess, and/or may be integrated with other assisted-driving processes.Skilled artisans will further recognize that a quicker initial setup maybe facilitated with respect to the process portions 1400A-1/1400A-3wherein blocks 1402-1438 may be replaced/augmented by a “generic” or“average” or “default” vehicle profile, and starting directly at userselection block 1440 in process portion 1400A-2 and using calibration inthe background (albeit with reduced accuracy for the user).

Turning now to FIG. 7, depicted therein is a block diagram of acomputer-implemented apparatus 700 that may be (re)configured and/or(re)arranged as a system, platform, node or element at an edge networklocation or a datacenter location according to an embodiment of thepresent patent disclosure. For example, apparatus 700 may be configuredin one embodiment as an edge compute location operative to execute thefunctionality of a base system for facilitating network-assistedscanning as set forth above. In another configuration, apparatus 700 maybe implemented as a cloud-based datacenter compute location interfacingwith an edge compute location and/or a plurality of sensing points orcollection agents. In an additional or alternative embodiment, apparatus700 may be configured as a network node operative as at least a portionof system 1300A-1300B for purposes of the present patent application.Skilled artisans will therefore recognize that apparatus 700 may besuitably configured to execute any of the processes, methods, and/orschemes set forth herein. Accordingly, one or more processors 702 may beoperatively coupled to various modules that may be implemented inpersistent memory e.g., memory 708, for executing suitable programinstructions or code portions with respect to one or more processes setforth in the present patent application for facilitatingnetwork-assisted scanning, environmental sensory data prioritization,CDS services, 2D/3D HD mapping, AR/MR-friendly data rendering, etc.Furthermore, depending on a particular configuration, a vehiclecalibration module 712, a vehicle position/location determination module716, and obstacle/hazard determination module 710 may be included aspart of apparatus 700. A Big Data analysis module 717 may be configuredto facilitate various data processing flows with respect to sensedenvironmental data, and possibly in conjunction with third-party datasources, as well as HAD/ADAS services, highly assisted location-basedservices, etc. At least for purposes of an embodiment of the presentapplication, “Big Data” may be used as a term for a collection of datasets so large and complex that it becomes virtually impossible toprocess using conventional database management tools or traditional dataprocessing applications in a scalable manner. Challenges involving BigData may include capture, curation, storage, search, sharing, transfer,analysis, and visualization, etc. Because Big Data available withrespect to user/vehicular activity data, surrounding environment data,etc. can be on the order of several terabytes to petabytes to exabytesdepending on the number of subscriber accounts, crowd-sourced collectionagent operators, vehicles, and UE devices as well as the vast number ofavailable third-party content sources, it becomes exceedingly difficultto work with using most relational database management systems foroptimizing, ranking, correlating and indexing activity data in typicalenvironments. In one embodiment, Big Data analytics platform 717 may beimplemented as a programming model framework for processing andgenerating large data sets with a parallel, distributed algorithmicengine that may be executed in a “massively parallel processing” (MPP)architecture with software running on a large number of servers (e.g., aserver farm). For example, a MapReduce programming platform may beconfigured as an implementation of the analytics service platform,either as part of apparatus 700 or separately associated therewith foranalyzing and correlating the data in order to generate data output insuitable formats for AR/MR rendition, 3D maps, etc. In oneimplementation, an example software/hardware framework may comprise acommon set of libraries and utilities needed by other modules, adistributed file system (DFS) that stores data on commodity machinesconfigured to provide a high aggregate bandwidth across the cluster, aresource-management platform responsible for managing compute resourcesin the clusters and using them in the execution of MapReduce-basedprogramming model. Broadly, an example MapReduce programming platformmay include an Input reader component, a Map function component, apartition function component, a compare function component, a Reducefunction component and an Output writer component, configured tointeroperate with various components, blocks/modules, and/or databasesof the example apparatus 700.

A data presentation module 718 may be operative with Big Data analysismodule 717 as well as one or more databases storing HD mapping data andAR/MR-friendly data (e.g., database 715) and road/ride quality (block706) may be provided for facilitating output data transfer to aplurality of consumption agents for purposes of an embodiment of thepresent invention. Depending on the configuration, various networkinterfaces (I/F) 714-1 to 714-K may be appropriately provided withrespect to effectuating communications with, inter alia,sensing/collection agents, consumption agents, vehicles, smartwearables, smart infrastructure elements/nodes, AR/MR/VR-capable enduser devices, edge compute locations, cloud-based datacenters,third-party databases, and the like.

FIG. 8 depicts a block diagram of a user equipment (UE) node or system800 having a collection agent according to an embodiment of the presentpatent disclosure. UE system 800 may be configured as a collection agent102-N, a data consumption agent 118-K, a sensing platform 303 and/oroperative in association with a vehicle 510-N according to one or moreembodiments set forth hereinabove, and may include appropriatehardware/software components and subsystems configured for performingany of the device-side processes (either individually or in anycombination thereof) with respect to generating, sensing and reportingvarious types of environmental data and receiving appropriate outputdata responses depending on implementation. One or moremicrocontrollers/processors 802 are provided for the overall control ofUE 800 and for the execution of various stored program instructionsembodied in a persistent memory 813 as a sensing/collection/consumptionagent as well as effectuating telematics sensing and data collectionwhere associated with a manual/autonomous vehicle, as appropriateclient-side applications that may be part of a memory subsystem 811 ofthe UE device. Controller/processor complex referred to by referencenumeral 802 may also be representative of other specialty processingmodules such as graphic processors, video processors, digital signalprocessors (DSPs), and the like, operating in association with suitablevideo and audio interfaces (not specifically shown). Appropriateinterfaces such as WiFi I/F modules 804 and satellite network I/Fmodules 806 involving tuners, demodulators, descramblers, etc. may beincluded for processing and interfacing with various short-rangewireless and satellite communications networks, e.g., asrepresentatively shown at reference numerals 878, 896, respectively.Example sensors may include camera(s)/charge-coupled devices (CCDs) 817,optical/sound/IR/GPS sensors 814, accelerometer/inclinometer/gyroscope816, as well as pressure, tactile, temperature sensors 810, etc.Additional sensory instrumentation may comprise gesturesensors/controllers, optical scanners, near-field communications (NFC)devices, head movement detectors, ocular movement trackers, facerecognition, and directional sensors such as solid-state compasses.Other I/O or interfaces such as a display interface 815, AR/MR interface820, USB/HDMI ports 818, Ethernet I/F 808, and wide area wirelessconnectivity interfaces 812 for connecting with suitable RANinfrastructures 813 are also provided. Although not specifically shown,a local storage may be included for storing various raw and/orpreprocessed environmental sensory data. UE 800 may also comprise is asuitable power supply block 822, which is may include AC/DC powerconversion to provide power for the device 800. It should be appreciatedthat the actual power architecture for the UE device may vary by thehardware platform used, e.g., depending upon the core SoC (System onChip), memory, analog front-end, analog signal chain components andinterfaces used in the specific platform, and the like.

One skilled in the art will recognize that various apparatuses,subsystems, scanning data processing functionalities/applications,HAD/ADAS systems, and/or 3D HD mapping and AR/MR-compatible datarendering functionalities, as well as the underlying networkinfrastructures set forth above may be architected in a virtualizedenvironment according to a network function virtualization (NFV)architecture in additional or alternative embodiments of the presentpatent disclosure. For instance, various physical resources, databases,CDS services, applications and functions executing an example networkenvironment of the present application may be provided as virtualappliances, machines or functions, wherein the resources andapplications are virtualized into suitable virtual network functions(VNFs) or virtual network elements (VNEs) via a suitable virtualizationlayer. Resources comprising compute resources, memory resources, andnetwork infrastructure resources are virtualized into correspondingvirtual resources wherein virtual compute resources, virtual memoryresources and virtual network resources are collectively operative tosupport a VNF layer, whose overall management and orchestrationfunctionality may be supported by a virtualized infrastructure manager(VIM) in conjunction with a VNF manager and an NFV orchestrator. AnOperation Support System (OSS) and/or Business Support System (BSS)component may typically be provided for handling network-levelfunctionalities such as network management, fault management,configuration management, service management, and subscriber management,etc., which may interface with VNF layer and NFV orchestrationcomponents via suitable interfaces.

Furthermore, at least a portion of an example network architecturedisclosed herein may be virtualized as set forth above and architectedin a cloud-computing environment comprising a shared pool ofconfigurable virtual resources. Various pieces of hardware/softwareassociated with the example systems/processes shown in, includingwithout limitation, FIGS. 3A-3C, 4A-4B, 6, 12, 13A-13B and 14A-1 to14A-3, 14B and 14C, etc. may be implemented in a service-orientedarchitecture, e.g., Software as a Service (SaaS), Platform as a Service(PaaS), infrastructure as a Service (IaaS) etc., with multiple entitiesproviding different features of an example embodiment of the presentinvention, wherein one or more layers of virtualized environments may beinstantiated on commercial off the shelf (COTS) hardware. Skilledartisans will also appreciate that such a cloud-computing environmentmay comprise one or more of private clouds, public clouds, hybridclouds, community clouds, distributed clouds, multiclouds andinterclouds (e.g., “cloud of clouds”), and the like.

In the above-description of various embodiments of the presentdisclosure, it is to be understood that the terminology used herein isfor the purpose of describing particular embodiments only and is notintended to be limiting of the invention. Unless otherwise defined, allterms (including technical and scientific terms) used herein have thesame meaning as commonly understood by one of ordinary skill in the artto which this invention belongs. It will be further understood thatterms, such as those defined in commonly used dictionaries, should beinterpreted as having a meaning that is consistent with their meaning inthe context of this specification and the relevant art and may not beinterpreted in an idealized or overly formal sense expressly so definedherein.

At least some example embodiments are described herein with reference toblock diagrams and/or flowchart illustrations of computer-implementedmethods, apparatus (systems and/or devices) and/or computer programproducts. It is understood that a block of the block diagrams and/orflowchart illustrations, and combinations of blocks in the blockdiagrams and/or flowchart illustrations, can be implemented by computerprogram instructions that are performed by one or more computercircuits. Such computer program instructions may be provided to aprocessor circuit of a general purpose computer circuit, special purposecomputer circuit, and/or other programmable data processing circuit toproduce a machine, so that the instructions, which execute via theprocessor of the computer and/or other programmable data processingapparatus, transform and control transistors, values stored in memorylocations, and other hardware components within such circuitry toimplement the functions/acts specified in the block diagrams and/orflowchart block or blocks, and thereby create means (functionality)and/or structure for implementing the functions/acts specified in theblock diagrams and/or flowchart block(s). Additionally, the computerprogram instructions may also be stored in a tangible computer-readablemedium that can direct a computer or other programmable data processingapparatus to function in a particular manner, such that the instructionsstored in the computer-readable medium produce an article of manufactureincluding instructions which implement the functions/acts specified inthe block diagrams and/or flowchart block or blocks.

As pointed out previously, tangible, non-transitory computer-readablemedium may include an electronic, magnetic, optical, electromagnetic, orsemiconductor data storage system, apparatus, or device. More specificexamples of the computer-readable medium would include the following: aportable computer diskette, a random access memory (RAM) circuit, aread-only memory (ROM) circuit, an erasable programmable read-onlymemory (EPROM or Flash memory) circuit, a portable compact discread-only memory (CD-ROM), and a portable digital video disc read-onlymemory (DVD/Blu-ray). The computer program instructions may also beloaded onto or otherwise downloaded to a computer and/or otherprogrammable data processing apparatus to cause a series of operationalsteps to be performed on the computer and/or other programmableapparatus to produce a computer-implemented process. Accordingly,embodiments of the present invention may be embodied in hardware and/orin software (including firmware, resident software, micro-code, etc.)that runs on a processor or controller, which may collectively bereferred to as “circuitry,” “a module” or variants thereof. Further, anexample processing unit may include, by way of illustration, a generalpurpose processor, a special purpose processor, a conventionalprocessor, a digital signal processor (DSP), a plurality ofmicroprocessors, one or more microprocessors in association with a DSPcore, a controller, a microcontroller, Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Array (FPGA) circuits, anyother type of integrated circuit (IC), and/or a state machine. As can beappreciated, an example processor unit may employ distributed processingin certain embodiments.

Further, in at least some additional or alternative implementations, thefunctions/acts described in the blocks may occur out of the order shownin the flowcharts. For example, two blocks shown in succession may infact be executed substantially concurrently or the blocks may sometimesbe executed in the reverse order, depending upon the functionality/actsinvolved. Moreover, the functionality of a given block of the flowchartsand/or block diagrams may be separated into multiple blocks and/or thefunctionality of two or more blocks of the flowcharts and/or blockdiagrams may be at least partially integrated. Furthermore, althoughsome of the diagrams include arrows on communication paths to show aprimary direction of communication, it is to be understood thatcommunication may occur in the opposite direction relative to thedepicted arrows. Finally, other blocks may be added/inserted between theblocks that are illustrated.

It should therefore be clearly understood that the order or sequence ofthe acts, steps, functions, components or blocks illustrated in any ofthe flowcharts depicted in the drawing Figures of the present disclosuremay be modified, altered, replaced, customized or otherwise rearrangedwithin a particular flowchart, including deletion or omission of aparticular act, step, function, component or block. Moreover, the acts,steps, functions, components or blocks illustrated in a particularflowchart may be inter-mixed or otherwise inter-arranged or rearrangedwith the acts, steps, functions, components or blocks illustrated inanother flowchart in order to effectuate additional variations,modifications and configurations with respect to one or more processesfor purposes of practicing the teachings of the present patentdisclosure.

Although various embodiments have been shown and described in detail,the claims are not limited to any particular embodiment or example. Noneof the above Detailed Description should be read as implying that anyparticular component, element, step, act, or function is essential suchthat it must be included in the scope of the claims. Reference to anelement in the singular is not intended to mean “one and only one”unless explicitly so stated, but rather “one or more.” All structuraland functional equivalents to the elements of the above-describedembodiments that are known to those of ordinary skill in the art areexpressly incorporated herein by reference and are intended to beencompassed by the present claims. Accordingly, those skilled in the artwill recognize that the exemplary embodiments described herein can bepracticed with various modifications and alterations within the spiritand scope of the claims appended below.

1. An Apparatus configured for facilitating geospatial sensing of anenvironment, the apparatus comprising: one or more processors and one ormore persistent memory modules having program instructions storedthereon which, when executed by one or more processors, perform thefollowing acts: collect environmental data by a collection agentoperating with a plurality of sensors configured to sense data relatingto the environment; prioritize the environmental data responsive to atleast one of a determination of cellular radio access network (RAN)resource capacity available to the collection agent, RAN transmissionresource capacity, cost of the cellular RAN resource capacity availableto the collection agent, and relative urgency of an event within theenvironment to which at least a portion of the environmental datapertains; and responsive to prioritizing the environmental data,transmit one or more pieces of the environmental data to at least one ofan edge network node and a cloud-based datacenter network node.
 2. Theapparatus as recited in claim 1, wherein the environment comprises ageographic area including one or more road segments, and further whereinthe environmental data comprises at least one of a location areaindication of the collection agent, motion data of a vehicle or a humanoperator sensed by the collection agent executing on a wireless userequipment (UE) device placed within or associated with the vehicle orthe human operator traversing at least a road segment and external datasensed by the sensors operating with the collection agent with respectto at least one of optical, infrared (IR), auditory, sonic, visual,aural, weather, and thermal data pertaining to a vicinity surroundingthe vehicle or the human operator disposed within the environment. 3.The apparatus as recited in claim 1, wherein the one or more processorsis further caused to transmit the environmental data to a dataconsumption agent via a peer-to-peer communication network path.
 4. Theapparatus as recited in claim 3, wherein the one or more processors isfurther caused to: determine whether any portion of the environmentaldata from the location area where the collection agent is disposed isneeded for updating a real-time high definition (HD) map of theenvironment; if so, further determine whether the collection agent isconnected via a suitable RAN having at least a threshold level of RANresource capacity including at least one of compute resources, storageresources and radio transmission resources; if the collection agent isconnected to the suitable RAN having at least a threshold level of RANresource capacity, transmit at least one or more pieces of theenvironmental data to the edge network node associated with the suitableRAN, the edge network node configured to execute an edge-based clientdata service; and otherwise, if the collection agent is not connected tothe suitable RAN having at least a threshold level of RAN resourcecapacity, transmit at least one or more pieces of the environmental datato the cloud-based datacenter node (352) configured to execute acloud-based client data service.
 5. The apparatus as recited in claim 4,wherein the edge-based client data service and the cloud-based clientdata service are connected to a public safety answering point (PSAP)network.
 6. (canceled)
 7. The apparatus as recited in claim 4, whereinthe one or more processors is further caused to facilitate processing ofthe environmental data by at least one of the edge network node and thecloud-based datacenter node for presenting output in at least one of anaugmented reality (AR) visualization mode, a mixed reality (MR)visualization mode, and a virtual reality (VR) visualization mode, theoutput comprising a rendition of at least a portion of the environment,wherein the rendition is operative to facilitate ambient-assisted motionwith respect to the vehicle or the human operator associated with thecollection agent.
 8. The apparatus as recited in claim 1, wherein atleast one of the edge network node and the cloud-based datacenter nodeare implemented as one or more virtual machines in a network functionvirtualization (NFV) architecture.
 9. The apparatus as recited in claim1, wherein at least one or more pieces of the environmental data istransmitted to the edge network node as a zero-rating service of asubscriber associated with the collection agent.
 10. A system configuredfor facilitating geospatial sensing of an environment, the systemcomprising: one or more processors; and a collection agent coupled tothe one or more processors and one or more persistent memory moduleshaving program instructions stored thereon which, when executed by theone or more processors, perform the following acts in association withone or more modules: collect environmental data by the collection agentin association with a plurality of sensors configured to sense datarelating to the environment; prioritize the environmental dataresponsive to at least one of a determination of cellular radio accessnetwork (RAN) resource capacity available to the collection agent, costof the cellular RAN resource capacity available to the collection agent,and relative urgency of an event within the environment to which atleast a portion of the environmental data pertains; and transmit one ormore pieces of the environmental data, responsive to prioritizing theenvironmental data, to at least one of an edge network node and acloud-based datacenter network node.
 11. The system as recited in claim10, wherein the environment comprises a geographic area including one ormore road segments, and further wherein the environmental data comprisesat least one of a location area indication of the collection agent,motion data of a vehicle or a human operator sensed by the collectionagent executing on a wireless user equipment (UE) device placed withinor associated with the vehicle or the human operator traversing at leasta road segment and external data sensed by the sensors operating withthe collection agent with respect to at least one of optical, infrared(IR), auditory, sonic, visual, aural, weather, and thermal datapertaining to a vicinity surrounding the vehicle or the human operatordisposed within the environment.
 12. The system as recited in claim 10,further comprising program instructions configured to facilitatetransmission of the environmental data to a data consumption agent via apeer-to-peer communication network path.
 13. The system as recited inclaim 12, further comprising program instructions configured to perform:determining whether any portion of the environmental data from thelocation area where the collection agent is disposed is needed forupdating a real-time high definition (HD) map of the environment; if so,further determining whether the collection agent is connected via asuitable RAN having at least a threshold level of RAN resource capacityincluding at least one of compute resources, storage resources and radiotransmission resources; if the collection agent is connected to thesuitable RAN having at least a threshold level of RAN resource capacity,transmitting at least one or more pieces of the environmental data tothe edge network node associated with the suitable RAN, the edge networknode configured to support an edge-based client data service; andotherwise, if the collection agent is not connected to the suitable RANhaving at least a threshold level of RAN resource capacity, transmittingat least one or more pieces of the environmental data to the cloud-baseddatacenter node configured to support a cloud-based client data service.14. The system as recited in claim 13, wherein the edge-based clientdata service and the cloud-based client data service (354) are connectedto a public safety answering point (PSAP) network.
 15. The system asrecited in claim 10, wherein at least one or more portions of theenvironmental data is transmitted using one or more wirelesstechnologies involving IEEE 802.11b, IEEE 802.11a, IEEE 802.11g, IEEE802.11p, HiperLan and HiperLan II standards, Wi-Max standard, OpenAirstandard, Bluetooth standard, an Enhanced Data Rates for Global Systemfor Mobile Communications (GSM) Evolution (EDGE) network technology, a3^(rd)/4^(th)/5^(th) Generation network technology, Long Term Evolution(LTE) technology, High-Speed Uplink Packet Access (HSUPA) technology,Evolved High-Speed Packet Access (HSPA) technology, an IntegratedDigital Enhanced Network (IDEN) technology, a Code Division MultipleAccess (CDMA) network technology, a Universal Mobile TelecommunicationsSystem (UMTS) network technology, a Universal Terrestrial Radio AccessNetwork (UTRAN) technology, an All-IP Next Generation Network (NGN)technology, an IP Multimedia Subsystem (IMS) technology, and a satellitetelephony network technology.
 16. The system as recited in claim 13,further comprising program instructions for facilitating output datapresentation in at least one of an augmented reality (AR) visualizationmode, a mixed reality (MR) visualization mode, and a virtual reality(VR) visualization mode, the output data comprising a rendition of atleast a portion of the environment, wherein the rendition is operativeto facilitate ambient-assisted motion with respect to the vehicle or thehuman operator associated with the collection agent.
 17. The system asrecited in claim 10, wherein at least one of the edge network node andthe cloud-based datacenter node are implemented as one or more virtualmachines in a network function virtualization (NFV) architecture. 18.The system as recited in claim 10, further comprising programinstructions for transmitting one or more pieces of the environmentaldata to the edge network node as a zero-rating service of a subscriberassociated with the collection agent.
 19. The system as recited in claim10, wherein the collection agent is configured to operate in associationwith a mobile communications device, a smart wearable device, a vehiclecontrol unit of a manual or autonomous vehicle, and one or moreInternet-of-Things (IoT)-enabled traffic and road condition sensorsdisposed within the environment.
 20. The system as recited in claim 19,wherein the autonomous vehicle is at least one of an autonomous landvehicle, an autonomous aerial vehicle, an autonomous amphibious vehicle,and an autonomous aquatic vehicle.
 21. (canceled)
 22. A method forfacilitating geospatial sensing of an environment, the methodcomprising: collecting environmental data by a collection agentoperating with a plurality of sensors configured to sense data relatingto the environment; prioritizing the environmental data responsive to atleast one of a determination of cellular radio access network (RAN)resource capacity available to the collection agent, RAN transmissionresource capacity, cost of the cellular RAN resource capacity availableto the collection agent, and relative urgency of an event within theenvironment to which at least a portion of the environmental datapertains; and responsive to prioritizing the environmental data,transmitting one or more pieces of the environmental data to at leastone of an edge network node and a cloud-based datacenter network node.